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On  the  Measureaent  of  Boslness  Perfonumce  In 
Strategy  Research:   A  Coaparlson  of  Approaches 

Abstract 

A  two-dimensional  classiflcatory  scheme  highlighting  ten  different 
approaches  to  the  measurement  of  business  performance  In  strategy  research 
Is  developed.  The  first  dimension  concerns  the  use  of  financial  versus 
broader  operational  criteria,  while  the  second  focuses  on  two  alternate 
data  sources,  I.e.,  primary  versus  secondary.  The  scheme  permits  the 
classification  of  an  exhaustive  coverage  of  measurement  approaches  and  Is 
useful  for  discussing  their  relative  merits  and  demerits.  Implications 
for  operatlonallzlng  business  performance  In  future  strategy  research  are 
discussed. 
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The  notion  of  performance  is  a  recurrent  theme  in  most  branches  of 
management,  including  strategic  management  and  is  of  interest  to  both 
academic  scholars  and  practicing  managers.  While  prescriptions  for 
improving  and  managing  organizational  performance  are  widely  available 
(see  for  instance  Nash,  1983),  the  academic  community  has  been  preoccupied 
with  discussions  and  debates  around  issues  of  terminology,  levels  of 
analysis  (i.e.,  individual,  work-unit,  or  organization  as  a  whole),  and 
conceptual  bases  for  assessment  of  performance  (see  Ford  &  Schellenberg, 
1982). 

Although  the  importance  of  the  performance  concept  (and  its  broader 
notion  of  organizational  effectiveness)  is  widely  recognized  (see 
especially  Campbell,  1977;  Connolly,  Conlon  &  Deutsch,  1980;  Goodman  & 
Pennings,  1977;  Hannan,  Freeman  &   Meyer,  1976;  Klrchoff,  1977;  Steers, 
1975;  1977;  Yuchtman  &  Seashore,  1967),  the  treatment  of  performance  in 
research  settings  is  perhaps  one  of  the  thorniest  of  issues  confronting 
the  academic  researcher  today.  With  the  volume  of  literature  on  this 
topic  continually  increasing,  there  appears  to  be  little  hope  of  reaching 
any  agreement  on  basic  terminology,  and  definitions.  Some  have  expressed 
considerable  frustration  with  this  concept.  In  a  recent  review,  Kanter 
and  Brinkerhoff  articulated  this  pessimism  as  follows:   "Some  leading 
scholars  have  expressed  impatience  with  the  very  concept  of 
"organizational  effectiveness,"  urging  researchers  to  turn  their  attention 
to  more  fruitful  fields"  (1981;  p.  321). 

The  laportance  of  Business  PerfoxBance  in  Strategic  Manageaent 

For  the  strategy  researcher,  the  option  to  move  away  from  defining 
(and  measuring)  performance  or  effectiveness  is  not  a  viable  one.  This 


-4- 


Is  because  performance  Improvement  is  at  the  heart  of  strategic  manage- 
ment. More  formally,  the  importance  of  business  performance  in  strategic 
management  can  be  argued  along  three  dimensions  —  viz,  theoretical, 
empirical,  and  managerial  (Cameron  &  Whetten,  1983a).  Theoretically,  the 
concept  of  business  performance  lies  at  the  center  of  strategic  manage- 
ment. Most  strategic  management  theories  either  implicitly  or  explicitly 
underscore  performance  implications,  since  performance  is  the  time  test 
of  any  strategy  (Schendel  &  Hofer,  1979).   Empirically,  most  strategy 
research  studies  employ  the  construct  of  business  performance  to  examine 
a  variety  of  strategy  content  and  process  issues  (see  Ginsberg  & 
Venkatraman,  1985  for  an  analytical  review  of  the  extent  to  which  the 
empirical  studies  reflect  the  performance  dimension).  The  managerial 
importance  of  business  performance  is  all  too  evident  in  the  many 
prescriptions  offered  for  performance  Improvement  (e.g.,  Nash,  1983). 
The  Increasing  volume  of  research  on  corporate  turnarounds  (e.g.,  Hofer, 
1980;  Ramanujam,  1984)  and  organizational  transitions  attests  to  the 
Interest  In  the  strategy  field  in  Issues  of  organizational  performance, 
adaptation  and  survival. 

drcoBscrlblng  the  Concept  of  Business  Perfoxaance 

The  Important  role  of  business  performance  in  strategic  management 
warrants  close  attention  to  the  topic  of  conceptualization  and  measurement 
of  business  performance.  However,  in  view  of  the  breadth  and  complexity 
of  the  topic,  we  circumscribe  the  scope  of  the  discussion  (a)  by  adopting 
the  perspective  of  the  field  of  strategic  management  field,  and  (b) 
focusing  on  measurement  issues. 
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The  Underlying  Perspective  of  Strategic  Management.  The  adoption 
of  a  disciplinary  focus  Is  deliberate  since  a  multl-dlsclpllnary 
perspective  Is  unlikely  to  move  the  discussion  on  measurement  beyond 
highlighting  the  fundamental  differences  In  terminology  and  assumptions 
existing  among  the  various  disciplines.  We  concur  with  Hofer  (1983)  that 
"...It  seems  clear  that  different  fields  of  study  will  and  should  use 
different  measures  of  organization  performance  because  of  the  differences 
In  their  research  questions."  (For  a  comprehensive  discussion  on  the 
"determinants"  of  organizational  performance  from  a  multl-dlsclpllnary 
perspective,  readers  are  directed  to  Lenz,  1981). 

Adopting  a  Measurement  Focus.   Similarly,  our  focus  on  measurement 
Issues  Is  a  conscious  one  and  Is  In  line  with  Steers'  work  on  measurement 
of  organizational  effectiveness.   Steers  argued  that  "a  meaningful  way  to 
understand  the  abstract  Idea  of  effectiveness  Is  to  consider  how 
researchers  have  operatlonallzed  and  measured  the  construct  In  their  work 
(1975;  p.  546)."   Although  any  treatment  of  measurement  Issues  Indepen- 
dent of  conceptual  and  definitional  Issues  Is  likely  to  be  Incomplete, 
there  exists  a  belief  among  some  researchers  that  a  critical  evaluation 
of  the  measurement  approaches  leads  to  an  impoved  understanding  of  the 
underlying  constructs  (Cameron  &  Whetten,  1983b;  Steers,  1975).  As 
Cameron  &  Whetten  noted,  "Constructs  such  as  intelligence,  motivation,  or 
leadership  — whose  construct  space,  by  definition,  also  is  not  bounded — 
have  been  better  understood  as  limited  aspects  of  their  total  meaning 
having  been  measured...  In  assessing  organization  effectiveness,  a 
similar  attack  seems  appropriate,  that  is,  to  concentrate  on  measuring 
limited  domains  of  the  construct  (1983b;  p.  267)." 

Thus,  this  paper  alms  to  classify  the  different  approaches  to  the 
measurement  of  business  performance  in  strategy  research  and  highlight 
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the  benefits  and  limitations  of  each  approach.  It  is  intended  to 
complement  recent  discussions  on  the  operatlonalization  of  key  strategic 
management  concepts  such  as  organizational  strategy  (Ginsberg,  1984;  Snow 
&  Hambrick,  1980),  business-level  strategies  (Hambrick,  1980),  diversifi- 
cation patterns  (Pitts  &  Hopkins,  1982)  and  organizational  slack 
(Bourgeois,  1981). 

Measureaent  of  Business  Perforaance;  A  (Hasslflcatory  Scheae 

An  important  issue  to  be  addressed  in  the  process  of  developing  a 
classificatory  scheme  for  highlighting  measurement  approaches  is  to 
delineate  the  domain  of  the  performance  concept.  More  specifically,  the 
question  is  whether  the  treatment  of  business  performance  should  be 
differentiated  from  the  overall  discussion  on  organizational 
effectiveness.   In  a  recent  comprehensive  discussion  on  organizational 
effectiveness,  Cameron  and  Whetten  noted  that 

"...As  a  construct,  organizational  effectiveness  is  similar  to 
an  unwrapped  terrain,  where  the  responsibility  lies  with  the 
investigator  to  chart  it.   (1983a;  pp.  19-20;  emphasis  added)." 

Our  view  in  this  paper  is  that  "business  performance,"  which  reflects  the 
perspective  of  strategic  management,  is  a  subset  of  the  overall  concept  of 
organizational  effectiveness.   Figure  1  provides  a  schematic  for  circum- 
scribing the  domain  of  business  performance  in  terms  of  the  scope  of 
coverage  in  the  concept's  domain. 

The  narrowest  conception  of  business  performance  centers  on  the  use 
of  simple  outcome-based  financial  indicators  that  are  assumed  to  reflect 


INSERT  FIGURE  1  ABOUT  HERE 
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the  fulfillment  of  the  economic  goals  of  the  firm.  We  refer  to  this 
concept  as  financial  performance,  which  has  been  the  dominant  model  in 
empirical  strategy  research  (Hofer,  1983).  Typical  of  this  approach 
would  be  to  examine  such  indicators  as  sales  growth,  profitability 
(reflected  by  ratios  such  as  return  on  investment,  return  on  sale,  and 
return  on  equity),  earnings  per  share,  etc.   In  addition,  reflecting  the 
popular  and  current  view  that  "market"  or  "value-based"  measurements  are 
more  appropriate  than  accounting-based  measures  (Hax  &  Majluf,  1984), 
some  strategy  studies  have  employed  such  measures  like  market-to-book  or 
stock-market  returns  and  its  variants  (e.g.,  Kuala,  1980;  Montgomery, 
Thomas  &  Kamath,  1984).  Extending  such  a  tradition,  one  can  possibly 
employ  measures  such  as  Tobin's  Q  —  the  ratio  of  the  market  value  of  a 
firm  to  the  replacement  cost  of  its  assets  (Lindberg  &  Ross,  1981). 
Nevertheless,  this  approach  remains  very  much  financial  in  its  orientation 
and  assumes  the  dominance  and  legitimacy  of  financial  goals  in  a  firm's 
system  of  goals. 

A  broader  conceptualization  of  business  performance  would  include 
emphasis  on  indicators  of  operational  performance  (i.e.,  non-financial) 
in  addition  to  indicators  of  financial  performance.  Under  this  framework, 
it  would  be  logical  to  treat  such  measures  as  market-share,  new  product 
introduction,  product  quality,  marketing  effectiveness,  manufacturing 
value-added,  and  other  measures  of  technological  efficiency  within  the 
domain  of  business  performance.  Similarly,  market  share  position,  widely 
believed  to  be  a  determinant  of  profitability  (Buzzell,  Gale  &  Sultan, 
1975)  would  be  a  meaningful  indicator  of  performance  within  this 
perspective.  The  inclusion  of  operational  performance  indicators  takes 
us  beyond  the  "black  box"  approach  that  seems  to  characterize  the 
exclusive  use  of  financial  indicators  and  focuses  on  those  key  operational 
success  factors  that  might  lead  to  financial  performance. 
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If  we  now  superimpose  multiple  and  conflicting  nature  of  organiza- 
tional goals  (Cameron  &  Whetten,  1983  a,b)  and  the  influence  of  multiple 
constituencies  or  "stakeholders",  we  move  towards  reflecting  the  writings 
on  organizational  effectiveness.   Perhaps  due  to  the  breadth  of  this 
discourse,  this  literature  is  plagued  with  debates  on  appropriate  models 
of  measurement  (Cameron  &  Whetten,  1983a,  Steers,  1975).  Although  we 
welcome  a  broader  conceptualization,  it  appears  that  most  strategy  studies 
have  restricted  their  focus  to  the  first  two  "circles"  in  Figure  1. 

While  business  performance  can  be  measured  using  the  financial 
Indicators,  operational  indicators,  or  both,  a  further  issue  in  its 
operationalization  is  the  sources  of  data.  The  sources  of  performance 
data  have  either  been  primary  (e.g.,  data  collected  directly  for  organiza- 
tions) or  secondary  (e.g.,  data  from  publically  available  records).  Using 
the  conceptualization  of  business  performance  (financial  versus 
operational  indicators)  and  data  sources  (primary  versus  secondary)  as  two 
basic,  but  different  concerns  in  the  overall  process  of  measuring  business 
performance,  a  four-celled  classificatory  scheme  (shown  in  Figure  2)  is 
developed. 

Figure  2  presents  four  "within  cell"  approaches  (numbered  1  through 
4)  and  six  "across  cell"  approaches  —  covering  two  adjacent  cells 
(labeled  A  through  F).  These  together  represent  ten  basic  alternate 
approaches  available  for  measuring  business  performance.  Measurement 
approaches  encompassing  more  than  two  cells  can  be  depicted  as  combina- 
tions of  these  basic  approaches  and  are  not  treated  separately.  The  ten 
basic  approaches  are  briefly  discussed  below  under  two  general  headings 
—  "within-cell"  and  "across-cell"  approaches. 


INSERT  FIGURE  2  ABOUT  HERE 
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Wthln-Cell  ^proches 

As  Figure  2  indicates,  four  of  the  ten  approaches  are  restricted  to 
operatlonallzatlons  within  a  particular  cell.  For  example,  in  Cell  1,  the 
operationalizing  scheme  for  business  performance  entails  the  use  of 
financial  performance  data  obtained  from  secondary  sources  (e.g..  Beard  & 
Dess,  1981;  Rumelt,  1974),  while  in  Cell  2,  the  focus  is  on  eliciting 
financial  data  directly  from  target  organizations  (e.g.,  ROI  in  PIMS  - 
based  studies).  In  Cells  3  and  4,  the  focus  Is  on  operational  indicators 
collected  from  secondary  sources  (e.g.,  market  share  data  in  Schendel  & 
Patton,  1978)  and  primary  sources  (e.g.,  data  on  market  share  positions 
in  PIMS-based  studies)  respectively.   It  is  readily  apparent  that  these 
four  approaches  have  a  narrow  perspective  on  the  notion  of  business 
performance.   Consequently  it  is  encouraging  to  note  that  only  a  few 
attempts  at  operationalizing  business  performance  in  strategy  research 
have  been  restricted  to  "within  cell"  approaches. 

Acxo88-Cell  ^proaches 

The  other  six  approaches,  labeled  (A)  through  (F)  represent  signi- 
ficant improvements  in  the  quality  of  operatlonallzatlons.  They  either 
(a)  reflect  a  broader  conceptualization  of  the  construct  space  of  business 
performance;  or  (b)  address  methodological  concerns  of  convergence  of 
operatlonallzatlons  across  distinct  methods.  These  are  discussed  below. 

Broader  Conceptualization  of  the  Construct  Space.  In  Figure  2  the 
approaches  denoted  by  (B)  and  (D)  reflect  theoretical  considerations  of 
employing  a  broader  construct  space.  An  illustration  of  the  measurement 
approach  (B)  can  be  seen  in  Schendel  and  Patton 's  (1978)  development  of  a 
simultaneous-equation  model  of  corporate  strategy.  There,  the  authors 
focused  on  mullple  performance  criteria  such  as  return  on  equity  (ROE) , 
market  share,  and  efficiency.  Financial  measures  (i.e.,  ROE)  and 
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operatlonal  measures  (I.e. ,  market  share  and  efficiency)  were  assembled 
from  the  COMPUSTAT  data  base,  a  widely-used  secondary  data  source 
for  strategy  research  (see  Glueck  &  Willis,  1979).  These  were  supple- 
mented by  other  secondary  data  on  the  brewing  industry. 

The  measurement  approach  denoted  as  (D)  is  similar  to  (B),  with  a 
focus  on  both  financial  and  operational  indicators,  but  the  difference  is 
that  data  are  obtained  from  primary  sources.  The  studies  by  Bourgeois 
(1980)  and  Gupta  and  Govindarajan  (1984),  which  collected  perceptual 
primary  data  on  both  financial  and  operational  Indicators  of  performance 
exemplify  this  approach.  Woo  and  Willard's  (1983)  use  of  the  PIMS  data 
base  to  explicate  the  underlying  dimensions  of  performance  is  another 
good  illustration  of  this  approach. 

Convergence  of  Methods.  The  approaches  denoted  as  (A)  and  (C) 
reflect  the  need  to  assess  method  convergence  across  different  data 
sources  or  methods.  Such  an  assessment  is  in  line  with  Campbell  and 
Fiske's  (1959)  development  of  the  Multi-Trait,  Multi-Method  (MTMM)  frame- 
work. When  the  data  obtained  from  primary  and  secondary  sources  are 
generally  in  agreement,  there  is  strong  support  for  the  operationallzation 
of  constructs,  especially  since  there  is  a  belief  that  managers  may  be 
biased  when  reporting  their  performance  levels.  An  Illustration  of  the 
use  of  the  approach  (A)  can  be  seen  in  a  study  by  Venkatraman  and 
Ramanujam  (1985).  Data  on  three  performance  Indicators  — sales  growth, 
profit  growth,  and  ROI —  were  collected  from  two  different  data  sources 
— primary  assessments  by  executives,  and  published  secondary  data. 
Analysis  of  the  data  Indicated  that  the  two  approaches  are  significantly 
correlated,  thus  providing  support  for  the  convergent  validity  of  the 
measures. 
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The  measurement  approach  labeled  (C)  Is  analogous  to  (A)  except 
that  the  focus  is  on  operational  Indicators  rather  than  financial 
indicators.  Again,  the  aim  is  to  assess  the  degree  of  method-convergence. 
However,  our  review  indicates  that  this  approach  has  not  yet  been  widely 
adopted.   Presumably,  this  is  because  of  the  difficulties  in  obtaining 
secondary  data  on  operational  indicators  like  market  share  that  are 
largely  free  of  aggregation  biases  and  definitional  problems. 

Other  Approaches.   The  measurement  approach  termed  (E)  calls  for 
collecting  data  on  financial  indicators  from  secondary  sources  and  on 
operational  indicators  from  primary  sources.  Such  an  approach  is 
appropriate  when  a  broader  conceptualization  of  business  performance  is 
needed  for  addressing  specific  research  questions,  but  data  on  financial 
performance  may  not  be  forthcomiong  from  primary  sourcas  due  to  reasons 
of  confidentiality  and  sensitivity.  On  the  other  hand,  measurement 
approach  (F),  the  converse  of  (E),  calls  for  financial  data  from  primary 
sources  and  operational  data  from  secondary  sources.  Although 
conceptually  a  feasible  approach,  it  is  unlikely  to  be  employed  for  the 
simple  reason  that  if  financial  data  are  forthcoming  from  primary  sources, 
it  is  also  likely  that  operational  data  could  be  obtained  from  the  same 
source.   Indeed,  obtaining  data  on  operational  indicators  from  other 
sources  would  raise  Issues  such  as  compatibility,  and  the  possible 
confounding  of  levels  of  analysis  (such  as  firm-level  and  SBU-level). 

Iq>llcatl<ms  for  Operatlonallzlng  Business  Perfonance 

The  development  of  both  descriptive  and  normative  theories  of 
strategy  (focused  on  both  content  and  process  issues)  must  continue  to  be 
firmly  rooted  in  explaining  differences  in  performance  results  (Schendel 
&  Hofer,  1979).  Thus  business  performance  is  an  important  concept  in 
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strateglc  management.  In  striving  towards  understanding  the  complex 
issues  involved  in  operatlonallzing  this  concept,  this  paper  has 
developed  a  classificatory  scheme  that  highlights  major  approaches  for 
measuring  business  performance.   Irrespective  of  whether  business 
performance  is  conceptualized  broadly  or  narrowly,  at  the  corporate-level 
or  at  the  business-level,  in  absolute  or  relative  terms,  the  available 
measurement  approaches  can  be  described  using  the  two  distinguishing 
characteristics,  viz.,  (1)  whether  the  concept's  domain  includes 
indicators  relating  to  financial,  operational,  or  both,  and  (11)  whether 
the  data  are  obtained  from  primary,  secondary,  or  both  sources. 

A  classificatory  scheme  such  as  the  one  presented  in  Figure  2  is 
useful,  in  two  ways.  One,  it  serves  as  basis  to  compare  and  contrast 
different  measurement  approaches.  Towards  this  end,  the  key  benefits  and 
limitations  of  the  ten  measurement  approaches  developed  in  the  previous 
section  are  summarized  in  Table  1.  In  addition,  the  table  contains  key 
methodological  Issues  and  illustrative  studies  adopting  these  approaches. 


INSERT  TABLE  1  ABOUT  HERE 


Two,  it  aids  the  development  of  a  set  of  recommendations  for 
strategy  researchers  to  consider  in  the  process  of  operatlonallzing 
business  performance.  Three  major  implications  can  be  derived  from  the 
classificatory  scheme  depicted  in  Figure  2,  and  the  benefits  and  limita- 
tions summarized  in  Table  1.   First,  operatlonallzing  business  performance 
using  the  "wlthin-cell"  approaches  should  be  avoided  to  the  extent 
possible.  However,  in  cases  where  only  the  "within  cell"  approaches  are 
feasible,  serious  attention  to  the  methodological  issues  listed  in  Table 
1  would  enhance  the  quality  of  operationalizatlon.   For  example,  if  the 
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study  focuses  on  firm-level  strategies  (as  opposed  to  SBU-level),  the  use 
of  secondary  sources  to  operatlonallze  financial  performance  may  be 
appropriate. 

The  second  Implication  relates  to  enlarging  the  construct  space  of 
business  performance,  and  the  third  Implication  calls  for  attention  to 
convergence  across  multiple  methods  of  data  collection.  These  Implica- 
tions raise  a  set  of  Issues  and  questions  —  which  are  discussed  below. 

Rnlarglng  the  Constmct  Space:  The  Issue  of  Dlaenslonallty 

The  adoption  of  measurement  approach  (B)  or  (D)  reflects  a  need  to 
conceptualize  the  domain  of  business  performance  In  teirms  broader  than 
economic  performance.  When  such  a  measurement  approach  Is  adopted.  It  Is 
necessary  to  be  particularly  sensitive  to  the  Issue  of  dimensionality  of 
business  performance.  This  Is  because  a  unl -dimensional  composite  of  a 
multl-dlmenslonal  concept  such  as  business  performance  tends  to  mask  the 
underlying  relationships  among  the  different  subdlmenslons.  Strategy 
researchers'  attention  has  been  repeatedly  drawn  to  the  conflicting 
nature  of  performance  dimensions  such  as  long-term. growthand  short-term 
profitability,  and  the  associated  problems  of  combining  them  into  one 
composite  dimension  of  performance.  Schendel  and  Pat ton  (1978)  highlight 
the  need  to  make  differential  resource  allocations  depending  upon  the 
desired  performance  outcome,  viz.,  ROE,  market  share,  or  efficiency, 
while  Klrchoff  and  Kirchoff  (1980)  provide  an  empirical  illustration  of 
the  dilemma  of  pursuing  differential  (and  sometimes,  conflicting) 
strategies  to  achieve  long-term  and  short-term  performance  results. 

The  dimensionality  issue  is  more  formally  addressed  in  a  recent 
study  by  Woo  and  Wlllard  (1983).  They  employed  a  factor-analytic 
framework  using  performance  data  from  the  PIMS  program.  An  analysis  of 
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14  Indicators,  covering  both  financial  and  operational  facets  of  business 
performance,  yielded  four  primary  dimensions  —  (i)  profitability/cash 
flow;  (ii)  relative  market  position;  (iii)  change  in  profitability  and 
cash  flow;  and  (iv)  revenue  growth.  While  Woo  and  Willard's  (1983)  study 
established  the  multi-dimensional  nature  of  business  performance  when 
conceptualized  using  financial  and  operational  indicators,  Venkatraman 
and  Ramanujam  (1985)  cautioned  that  even  within  the  domain  of  financial 
performance,  indicators  such  as  sales  growth,  net  income  growth,  and  ROI 
should  not  be  combined  to  form  one  composite  dimension,  as  they  seem  to 
reflect  distinct  dimensions. 

The  implication  of  the  above  discussion  is  that  researchers  should 
collect  data  on  indicators  of  business  performance  either  using  an  a 
priori  classification  which  recognizes  the  dimensionality  issue,  or  they 
should  explicitly  test  the  dimensionality  of  their  conceptualization  of 
business  performance.  This  is  especially  critical  in  view  of  the  enlarged 
domain  of  business  performance  discussed  here  is  adopted  and  both  finan- 
cial and  operational  aspects  of  business  performance  are  included  using 
measurement  approaches  (B)  or  (D). 

Aasesslng  Method  ConTergence :  Chooslns  an  Efficient  Method 

The  adoption  of  measurement  approach  (A)  or  (C)  by  using  different 
data  sources  implies  a  fundamental  motivation  to  examine  convergence 
between  data  from  alternate  sources.  A  key  data-analytic  issue,  then,  is: 
How  does  one  evaluate  the  degree  of  convergence  of  operationalization? 
The  most  common  approach  is  based  on  the  magnitude  and  the  level  of 
statistical  significance  of  the  correlation  between  the  two  sets  of  data. 
Such  an  approach  underlies  Campbell  and  Fiske's  (1959)  criterion  for 
convergent  validity  in  the  MTMM  matrix  that  the  correlation  (i.e.. 
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validity)  coefficients  should  be  "sufficiently  large"  and  statistically 
different  from  zero.  However,  an  MTMM  analysis  does  not  address  the 
relative  efficiency  of  alternate  methods.  This  Is  Important  since 
researchers  examining  convergence  between  measures  to  assess  the  quality 
of  their  operatlonallzatlons  may  elect  to  use  one  or  the  other,  but  not 
necessarily  both,  when  subsequently  examining  theoretical  relationships. 

The  relative  efficiency  or  interchangeablllty  of  alternate  measures 
can  be  assessed  in  at  least  two  ways.  One  calls  for  checking  whether  the 
two  measures  are  not  only  correlated  but  also  proportional  to  each  other, 
since  a  positive  correlation  between  the  two  measures  Is  a  necessary  but 
not  a  sufficient  condition  for  the  interchangeablllty  of  measures  (Smyth, 
Boyes  &  Peseau,  1975).  The  other  method  calls  for  employing  the  two 
measures  as  alternate  operatlonallzatlons  with  the  same  measurement  model. 
By  evaluating  the  measurement  model  with  the  restricted  condition  of 
equivalence  of  measures  and  comparing  it  with  an  unrestricted  model,  the 
appropriateness  of  the  equivalence  criterion  can  be  ascertained  (see 
Bagozzl,  1980  for  a  discussion). 

If  the  interchangeablllty  condition  is  not  satisfied,  the  researcher 
is  faced  with  the  task  of  deciding  between  alternate  operatlonallzatlons. 
While  one  could  base  the  decision  on  one  gut-feel  and  intuition  about  the 
data  quality,  a  more  systematic  approach  is  to  examine  the  level  of 
measurement  error  in  the  different  operatlonallzatlons.   This  can  be 
analyzed  using  structural  equation  models  (see  Joreskog  &  Sorbum,  1979 
for  an  overview  of  this  approach  and  Bagozzl,  1980  for  illustrations 
comparing  levels  of  measurement  error  in  different  operatlonallzatlons). 
An  illustration  which  is  more  germane  to  the  present  discussion  can  be 
seen  in  Venkatraman  and  Ramanujam  (1985)  —  where  the  structural  equation 
approach  was  used  to  establish  the  relative  superiority  of  data  from  one 
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source  over  corresponding  data  from  the  other  source.  A  systematic 
evaluation  of  method  efficiency  was  accomplished  In  that  study  for 
different  dimensions  of  business  economic  performance. 

Non-Convergence  as  a  Source  of  Opportunity.  While  the  above 
discussion  Is  relevant  when  data  across  multiple  methods  do  converge  at 
some  acceptable  level  of  association,  what  Is  the  researcher  to  Infer  If 
convergence  between  dlfferen  operatlonallzatlons  Is  not  observed?  Does 
that  Imply  that  the  performance  data  at  hand  Is  "Inferior",  or  are  the 
differences  attributable  to  aspects  such  as  the  definition  of  performance 
or  the  level  of  aggregation?  As  Jlck  noted,  "When  different  measures 
yield  dissimilar  results,  they  demand  that  the  researcher  reconcile  the 
differences  somehow.   In  fact,  divergence  can  often  turn  out  to  be  an 
opportunity  for  enriching  the  explanation"  (1979;  p.  607). 

Researchers  can  pursue  at  least  two  different  courses  at  this  stage. 
One  Is  the  methodological  option  of  decomposing  the  variance  In  measure- 
ment Into  trait,  method,  and  random  error  components  (Joreskog  &  Sorbum, 
1979),  which  Is  useful  for  testing  If  systematic  biases  due  to  method 
differences  are  contributing  to  the  observed  lack  of  convergence.  The 
other  option  Is  a  conceptual  one,  which  attempts  to  ascertain  see  If 
definitional  differences  or  aggregation  problems  could  have  contributed  to 
the  observed  result.  By  pursuing  both  options  simultaneously,  significant 
Improvements  can  be  made  In  subsequent  efforts  at  operatlonallzlng 
business  performance. 

Although  problems  of  a  conceptual  nature  continue  to  underly  much 
of  the  discussion  on  organizational  performance.  Its  use  as  a  key 
construct  In  strategy  research  studies  has  continued  unabated.   Strategic 
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management  researchers,  in  their  quest  for  establishing  performance 
implications  of  strategic  conduct  of  businesses,  continue  to  measure 
business  performance  using  a  wide  array  of  operationalizing  schemes. 
Motivated  by  the  belief  that  systematic  approaches  to  measurement 
approaches  are  likely  to  lead  to  superior  operationalizations,  this  paper 
classified  and  highlighted  the  advantages  and  limitations  of  different 
measurement  approaches.  In  addition,  specific  data-analytic  issues  and 
their  implications  for  operationalizing  business  performance  are 
highlighted. 
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ILLUSTRATIVE 
STUDIES 

REFERENCE 

-market-share 
data  in 
Schendel  and 
Patton(197e) 

-market-share 
data  in  PIHS- 
based  studies 

Venkat  raaan 
and  Rananujan 
(1985 
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(a)  Use  industry  •■  the  refer- 
ence point  for  developing 
aeasures , 

(b)  Attempt  to  define  concepts 
such  as  market-share, effi- 
ciency, etc.  aa  consistently 
as  possible  across  industries. 

(a)  Choose  target  respondents 
based  on  specific  criteria 
(position,  function,  etc.) 

(b)  Use  multiple  respondents  to 
examine  the  extent  of  sys- 
tematic bias  as  well  as  aln- 
laize  aeasurement  error. 

(a)  Check  for  definitional  con- 
sistency; 

<b)  Identify  reaaons  for  non- 
convergence,  if  observed  and 
reconceptualize  the  concept, 
if  necessary. 

(c)  Examine  subs t it ut sbil it y/ 
interchange  ability  of  one 
measure  for  the  other. 

(d)  Use  Industry  referrent  mea- 
sures in  multi  industry 

St  ud les . 

(e)  Evaluate  the  feasibility  of 
using  stock-market  measures 
or  KOVA 

z 
o 

< 
X 

(a)  Problems  of  data 
availability  on 
various  indicators 
to  develop  the  re- 
quisite measures; 

(b)  Some  operational 
data  aay  be  indus- 
try-specific and 
■ay  not  lend  it- 
self to  multi  in- 
dustry studies. 

(c)  Relationship  to 
financial  perfor- 
mance not  known. 

(a)  Data  is  likely  to 
be  biased; 

(b)  Relstlonship  to 
financial  perfor- 
mance not  known. 

(a )  Compd  cible  data 
fruui  iwu  different 
data  sources  may 
not  be  readily 
available; 

(b)  Cannot  be  meaning- 
fully done  when  re- 
search studies  are 
designed  st  the  SBU 
level  due  to  pro- 
leuB  of  obtaining 
secondary  data; 

(c)  Operational  aspects 
of  performance  are 
not  covered. 

M 

H 
1.^ 
lb 
U 

z 
u 

OS 

(a)  Provides  perfor- 
mance data  when 
financial  data 
aay  either  be  not 
available  or  aay 
be  inappropriate. 

(a)  Provides  some 
basis  to  include 
considerations  of 
performance  in  the 
research  design; 

(b)  Less  likely  to  be 
influenced  by 
reasons  of  confi- 
dentiality ,sensl- 
t  Ivity  ,  etc. 

(a)  Provides  scope  for 
d^^ebsliiB  CO  11  ver- 
ge II I  validity  to 
enhance  the 
"quality"  of  mea- 
surement . 
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Operat lonal 
data  froa 
secondary 
sources 

Operat lonal 
data  from 
primary 
sources 

Financial 
data  f  rem 
both  primary 
and  secondary 
sources 
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